Is it Living? Insights from Modeling Event-Oriented, Self-Motivated, Acting, Learning and Conversing Game Agents

A cognitive architecture is presented, which combines insights from artificial intelligence with cognitive psychology, biology, and linguistics. Using a Super Mario clone, we equipped the simulated agents with (i) motivational behavioral systems, (ii) reasoning and planning capabilities, (iii) event-based schema learning and sensorimotor exploration, and (iv) speech comprehension and generation mechanisms. The motivational system activates goal events to maintain internal homeostasis. To invoke selected events, hierarchical action planning and control unfolds both on an event-schematic and a sensorimotor level. Schema learning is based on the detection of event changes, which are not predicted by the basic sensorimotor forward model. Language is comprehended and generated using context-free grammars linked to the schema-based knowledge structure. The work offers an approach to develop and thus to ground conceptual, semantic world knowledge in sensorimotor interactions and to couple this knowledge with a language to generate and comprehend language about the agent’s virtual world meaningfully.